Using voice commands to query and control external devices, process or services has become a reality thanks to the rise of digital personal assistant applications. However, these applications are still very limited in terms of functionalities, abilities, and degrees of automation.
In the open world of Internet of Things (IoT), all devices and services are connected and most of them are programmatically accessible, by exposed application programming interfaces (APIs) for basic operations such as reading and executing, or more advanced ones such as push notifications for time critical missions. Making these APIs accessible and interconnected via natural speech or text without requiring much, if any, programming knowledge from users is one of the goals of next generation human computer interaction efforts. Challenges arise when a system needs to identify multiple intentions from various domains in a single complex sentence. It is also challenging to fulfill these complex natural language instructions with underlying API calls.
There are services supporting these tasks, however, all of them require a long process of users specifying exactly what services are intended and a relatively significant and burdensome amount of configuration. For instance, a conventional service requires a process of several steps for a user to fill out to generate a desired automatic rule. Usually this process takes minutes to tens of minutes.
On the other hand, the desire to automate these tasks and make them available to general users without requiring significant and in some cases expert programming knowledge has been a motivation for the development of various personal assistant applications and devices (e.g., Google Home, Apple Siri, Microsoft Cortana, Amazon Echo), all of which provide a text or speech interface to perform simple tasks for users. However, the abilities of these personal assistant applications are still very limited to both very simple commands and the relatively small ecosystem built by their providers.